Applications of DEC-MDPs in multi-robot systems

International audienceOptimizing the operation of cooperative multi-robot systems that can cooperatively act in large and complex environments has become an important focal area of research. This issue is motivated by many applications involving a set of cooperative robots that have to decide in a decentralized way how to execute a large set of tasks in partially observable and uncertain environments. Such decision problems are encountered while developing exploration rovers, teams of patrolling robots, rescue-robot colonies, mine-clearance robots, et cetera.In this chapter, we introduce problematics related to the decentralized control of multi-robot systems. We rst describe some applicative domains and review the main characteristics of the decision problems the robots must deal with. Then, we review some existing approaches to solve problems of multiagent decen- tralized control in stochastic environments. We present the Decentralized Markov Decision Processes and discuss their applicability to real-world multi-robot applications. Then, we introduce OC-DEC-MDPs and 2V-DEC-MDPs which have been developed to increase the applicability of DEC-MDPs

DECENTRALIZED MULTI-ROBOT PLANNING TO EXPLORE AND PERCEIVE

In a recent French robotic contest, the objective was to develop a multi-robot system able to autonomously map and explore an unknown area while also detecting and localizing objects. As a participant in this challenge, we proposed a new decentralized Markov decision process (Dec-MDP) resolution based on distributed value functions (DVF) to compute multi-robot exploration strategies. The idea is to take advantage of sparse interactions by allowing each robot to calculate locally a strategy that maximizes the explored space while minimizing robots interactions. In this paper, we propose an adaptation of this method to improve also object recognition by integrating into the DVF the interest in covering explored areas with photos. The robots will then act to maximize the explored space and the photo coverage, ensuring better perception and object recognition

Simultaneous Auctions for "Rendez-Vous" Coordination Phases in Multi-robot Multi-task Mission

International audienceThis paper presents a protocol that permits to automatically allocate tasks, in a distributed way, among a fleet of agents when communication is not permanently available. In cooperation settings when communication is available only during short periods, it is difficult to build joint policies of agents to collectively accomplish a mission defined by a set of tasks. The proposed approach aims to punctually coordinate the agents during "Rendezvous'' phases defined by the short periods when communication is available. This approach consists of a series of simultaneous auctions to coordinate individual policies computed in a distributed way from Markov decision processes oriented by several goals. These policies allow the agents to evaluate their own relevance in each task achievement and to communicate bids when possible. This approach is illustrated on multi-mobile-robot missions similar to distributed traveling salesmen problem. Experimental results (through simulation and on real robots) demonstrate that high-quality allocations are quickly computed

COACHES Cooperative Autonomous Robots in Complex and Human Populated Environments

Public spaces in large cities are increasingly becoming complex and unwelcoming environments. Public spaces progressively become more hostile and unpleasant to use because of the overcrowding and complex information in signboards. It is in the interest of cities to make their public spaces easier to use, friendlier to visitors and safer to increasing elderly population and to citizens with disabilities. Meanwhile, we observe, in the last decade a tremendous progress in the development of robots in dynamic, complex and uncertain environments. The new challenge for the near future is to deploy a network of robots in public spaces to accomplish services that can help humans. Inspired by the aforementioned challenges, COACHES project addresses fundamental issues related to the design of a robust system of self-directed autonomous robots with high-level skills of environment modelling and scene understanding, distributed autonomous decision-making, short-term interacting with humans and robust and safe navigation in overcrowding spaces. To this end, COACHES will provide an integrated solution to new challenges on: (1) a knowledge-based representation of the environment, (2) human activities and needs estimation using Markov and Bayesian techniques, (3) distributed decision-making under uncertainty to collectively plan activities of assistance, guidance and delivery tasks using Decentralized Partially Observable Markov Decision Processes with efficient algorithms to improve their scalability and (4) a multi-modal and short-term human-robot interaction to exchange information and requests. COACHES project will provide a modular architecture to be integrated in real robots. We deploy COACHES at Caen city in a mall called “Rive de l’orne”. COACHES is a cooperative system consisting of ?xed cameras and the mobile robots. The ?xed cameras can do object detection, tracking and abnormal events detection (objects or behaviour). The robots combine these information with the ones perceived via their own sensor, to provide information through its multi-modal interface, guide people to their destinations, show tramway stations and transport goods for elderly people, etc.... The COACHES robots will use different modalities (speech and displayed information) to interact with the mall visitors, shopkeepers and mall managers. The project has enlisted an important an end-user (Caen la mer) providing the scenarios where the COACHES robots and systems will be deployed, and gather together universities with complementary competences from cognitive systems (SU), robust image/video processing (VUB, UNICAEN), and semantic scene analysis and understanding (VUB), Collective decision-making using decentralized partially observable Markov Decision Processes and multi-agent planning (UNICAEN, Sapienza), multi-modal and short-term human-robot interaction (Sapienza, UNICAEN

Punctual versus continuous auction coordination for multi-robot and multi-task topological navigation

International audienceThis paper addresses the interest of using Punctual versus Continuous coordination for mobile multi-robot systems where robots use auction sales to allocate tasks between them and to compute their policies in a distributed way. In Continuous coordination, one task at a time is assigned and performed per robot. In Punctual coordination, all the tasks are distributed in Rendezvous phases during the mission execution. However , tasks allocation problem grows exponentially with the number of tasks. The proposed approach consists in two aspects: (1) a control architecture based on topo-logical representation of the environment which reduces the planning complexity and (2) a protocol based on Sequential Simultaneous Auctions (SSA) to coordinate Robots' policies. The policies are individually computed using Markov Decision Processes oriented by several goal-task positions to reach. Experimental results on both real robots and simulation describe an evaluation of the proposed robot architecture coupled wih the SSA protocol. The efficiency of missions' execution is empirically evaluated regarding continuous planning

Calcul distribué de politiques d'exploration pour une flotte de robots mobiles

National audienceCe papier présente une architecture multirobots permettant une allocation automatique de plusieurs objectifs sur une flotte de robots. Le challenge consiste à rendre des robots autonomes pour réaliser coopérativement leur mission sans qu'un plan soit prédéfini. Cette architecture, appelée PRDC, est basée sur 4 modules (Perception, Représentation, Délibération et Contrôle). Nous nous intéressons plus particulièrement au module de délibération en considérant le problème des voyageurs de commerce coopératifs dans un environnement incertain. L'objectif des robots est alors de visiter un ensemble de points d'intérêt représentés dans une carte topologique stochastique (Road-Map). Le processus proposé pour la construction des politiques collaboratives est distribué. Chaque robot calcule ses politiques individuelles possibles de façon à négocier collectivement l'allocation des points d'intérêt entre les membres de la flotte. Enfin, l'approche est évaluée via un important nombre de simulation

Map Partitioning to Approximate an Exploration Strategy in Mobile Robotics

International audienceIn this paper, an approach is presented to automatically allocate a set of exploration tasks between a fleet of mobile robots. The approach combines a Road-Map technique and Markovian Decision Processes (MDPs). The addressed problem consists of exploring an area where a set of points of interest characterizes the main positions to be visited by the robots. This problem induces a long term horizon motion planning with a combinatorial explosion. The Road-Map allows the robots to represent their spatial knowledge as a graph of way-points connected by paths. It can be modified during the exploration mission requiring the robots to use on-line computations. By decomposing the Road-Map into regions, an MDP allows the current group leader to evaluate the interest of each robot in every single region. Using those values, the leader can assign the exploration tasks to the robots

COACHES: an assistance multi-robot system in public areas

In this paper, we present a robust system of self-directed autonomous robots evolving in a complex and public spaces and interacting with people. This system integrates highlevel skills of environment modeling using knowledge-based modeling and reasoning and scene understanding with robust image and video analysis, distributed autonomous decisionmaking using Markov decision process and Petri-Net planning, short-term interacting with humans and robust and safe navigation in overcrowding spaces. This system has been deployed in a variety of public environments such as a shopping mall, a center of congress and in a lab to assist people and visitors. The results are very satisfying showing the effectiveness of the system and going beyond just a simple proof of concepts

Vector-Value Markov Decision Process for multi-objective stochastic path planning

International audienceThe problem of path planning in stochastic environments where the shortest path is not always the best one is a challenging issue required in many real-world applications such as autonomous vehicles, robotics, logistics, etc. . . . In this paper, we consider the problem of path planning in stochastic environments where the length of the path is not the unique criterion to consider. We formalize this problem as a multi-objective decision-theoretic path planning and we transform this latter into 2VMDP (Vector-Valued Markov Decision Process). We show, then, how we can compute a policy balancing between different considered criteria. We describe different techniques that allow us to derive an optimal policy where it is hard to express the expected utilities, rewards and values with a unique numerical measure. Firstly, we examine different existing approaches based on preferences and we define notions of optimality with preferred solutions and secondly we present approaches based on egalitarian social welfare techniques. Finally, some experimental results have been developed to show the feasibility of the approach and the benefit of this approach on the single-objective techniques